Biaxial nematic phases in ultracold dipolar Fermi gases 
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Ultracold dipolar Fermi gases represent relatively unexplored, strongly correlated systems arising 
from long-range and anisotropic interactions. We demonstrate the possibility of a spontaneous 
symmetry breaking biaxial phase in these systems, which may be realized in, e.g., gases of ultracold 
polar molecules or strongly magnetic atoms. This biaxial nematic phase is manifest in a spontaneous 
distortion of the Fermi surface perpendicular to the axis of polarization. We describe these dipolar 
interaction induced phases using Landau Fermi liquid theory. 

PACS numbers: 03.75.Ss, 05.30.Fk, 75.80.+q, 71.10.Ay 



Recent experimental advances toward the creation 
of degenerate fermionic rovibronic ground state polar 
molecules, such as "'"K^^Rb jlj, and highly magnetic 
fermionic atoms, such as ^^Cr [2E], ^'^^Er gj, and ^^^Dy 
or ""^^^Dy, present new avenues for realizing states of 
strongly correlated matter in ultracold atomic and molec- 
ular systems. A degenerate dipolar Fermi gas (DDFG) is 
more strongly coupled and has longer range interactions 
than other (neutral) atomic systems, and it is therefore 
reasonable to expect that these systems will exhibit novel 
and exotic behaviors. Recent theoretical studies have al- 
ready explored BCS-like superfluidity in DDFG systems 
(see Ref. [5] and references therein). 

We explore the possible occurrence in homogeneously 
trapped DDFGs of quantum nematic phases; phenomena 
heretofore only observed in strongly correlated electronic 
systems |6l |7l and which have been shown to occur in 
models based on the breakdown of Fermi liquid (FL) the- 
ory at a Pomeranchuk instability j2] as well as in lattice 
models [inilll]- In a nematic state, the simplest example 
of quantum liquid crystal (QLC) order |TH|, the fermionic 
gas becomes spontaneously anisotropic — without the ac- 
tion of an external field — while maintaining overall ho- 
mogeneity. Other inhomogeneous QLC phases, such as 
the smectic (stripe) phase, are both possible and Hkely 
to occur in the strong dipolar coupling regime, though 
are not considered here. Quantum liquid crystal phases 
may also occur in highly magnetic atomic systems in op- 
tical lattices |13] and in population unbalanced Fermi 
gases p3] . 

We show in this work that a quantum nematic phase is 
possible in suSiciently strongly coupled, homogeneously 
trapped DDFGs. More specifically, we consider DDFGs 
in a fixed external polarizing field pointing along the z- 
axis. We consider an infinite and homogeneous 3D sys- 
tem in which the fermionic atoms or molecules interact 
via the dipolar interaction that, unlike contact in- 
teractions, has a long-range falloff oc r~'^ . In the pres- 
ence of a polarizing field, the interaction itself becomes 
anisotropic in real space with a d-wave symmetry; the 
dipole-dipole interaction (DDI) is repulsive or attractive 
depending on the spatial configuration of dipoles. Since 



the DDI is explicitly anisotropic, the Hamiltonian (and 
ground state) of the uniformly polarized DDFG must 
have an explicitly broken rotational invariance, with the 
polarizing field providing the preferred axis. Neverthe- 
less, the Hamiltonian is invariant under rotations about 
this fixed axis, and it is this remaining symmetry that is 
spontaneously broken in the biaxial nematic phase. 

While DDFGs with interactions in the particle-particle 
channel have been previously studied [5] [TS] [161 IT?] , 
the particle-hole (density) channel is relatively unex- 
plored [T^. In the case of fermionic dipoles aligned by 
an external field, the explicit broken symmetry of the 
DDI cans es the coupling of even and odd orbital angu- 
lar momentum channels |19j . More significantly for QLC 
physics, we show that an important feature of the DDI 
in 3D is the presence of an effective attractive particle- 
hole interaction in the d-wave angular momentum chan- 
nels, which triggers a Pomeranchuk instability leading 
to a nematic phase of the FL. While these phases are 
difficult to observe and study in electronic systems, ul- 
tracold DDFGs provide clean, experimentally realizable 
systems whose interaction strengths in the, e.g., / = and 
I = 2, angular momentum channels can be comparable 
by changing external parameters such as the polarizing 
field and trap aspect ratio. 

We now apply Landau FL theory to explore the 
particle-hole channel instabilities of a DDFG, and in do- 
ing so, reveal the possibility for observing a nematic state 
in the form of a spontaneous ccy-symmetry breaking. The 
starting point of the Landau FL theory is the existence of 
a Fermi surface (FS) representing the ground state of the 
FL in the absence of quasiparticle excitations . In a 
FL, fermionic quasiparticles have residual interactions in 
the forward scattering channel parametrized by a set of 
coupling constants, the Landau parameters of the FL. In 
a 3D system, such interactions can occur in any angular 
momentum channel. This standard picture is known to 
be a valid description for interacting Fermi systems such 
as "^He and most metals. 

The order parameter of a nematic state is a trace- 
less symmetric tensor [21]. In a 3D interacting Fermi 
system, such as the DDFG, we can form a nematic or- 
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der parameter as a bilinear of the Fermi fields, Qij = 
-^ip^{x) {pidj — |V^(5ij) tp{x), which is both symmetric 

and traceless [3]. Here, i,j = x,y,z and tp{x) is the sec- 
ond quantized fermion operator at position x. The 3x3 
symmetric tensor Qij has two independent eigenvalues, 
Qi and Q2- If Qi = Q2, then the nematic state is uniax- 
ial; otherwise it is biaxial. Since the Hamiltonian of the 
polarized DDFG has an explicitly broken rotational in- 
variance (due to the polarizing field) , Qij ^ in a uniax- 
ial state. In this case, the FS of the gas is not spherically 
symmetric but must instead have (at least) a uniaxial 
distortion: the FS is an ellipsoid oriented along the di- 
rection of polarization. In a uniformly polarized DDFG, 
polarization effects enter only through the anisotropy of 
the resulting interaction. Hence, the uniaxial distortion 
of the FS must be a (generally monotonic) function of the 
polarization. Indeed, Hartree-Fock (variational) calcula- 
tions |18j — accurate in the weak coupling regime — reveal 
this uniaxial FS distortion of a polarized DDFG. 

Since the FS distortions are described by particlc-holc 
condensation in specific angular momentum channels, the 
general tensor order parameter Qij can be recast in terms 
of its angular momentum components ug^m , related to the 
fermion occupation numbers by: 
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where is the Fourier transform of the Fermi field 'ip{x), 
t > and \m\ < £; Y«.m(fc) arc the familiar spherical 
harmonics. Each order parameter ug^m measures a dis- 
tortion mode of the Fermi surface. Among them, U2,o ^^nd 
U2,2 = U2 _2 are of main interest, as they are order pa- 
rameters for a uniaxial and biaxial nematic phase, respec- 
tively: |u2,o| oc (Qi-|-(52)/2 and |u2,2| oc \Qi-Q2\- More 
generally, Eq. ([T]) represents moments of the Fermi-Dirac 
momentum distribution function that is anisotropic in 
these nematic phases. As a consequence, in a trapped 
system, the density profile will distort — an effect, for 
the uniaxial case, seen in the bosonic dipolar conden- 
sates [12]. 

Unlike the uniaxial state, which is present in a uni- 
formly polarized DDFG for all polarizing fields, the bi- 
axial state is accessible only for fields of sufficient mag- 
nitude. Whether or not this phase exists in a given 
dipolar gas depends sensitively on the particular Lan- 
dau parameters, which is reliably calculated numerically 
(by intensive quantum Monte Carlo simulations) or an- 
alytically in the weak dipole limit. The latter is defined 
by the condition that the dimensionless coupling con- 
stant A = ncP/ep is small, where n is the particle den- 
sity, d is the electric or magnetic dipole moment, and 
ep = h^{6n^n)^/^ /2m is the Fermi energy of the undis- 
torted FS. The Fermi temperature Tp is ep = ksTp. 
Our mean field analysis predicts that a biaxial phase may 
arise in DDFGs for A ~ 1. 



Consider a uniformly polarized DDFG described by the 
Hamiltonian: 
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where f{q) = ^^^(3cos^ ^ij~l) is the DDI in momentum 
space and 9q is the angle between q and z. Within the 
mean field (Hartrcc-Fock) approach, the Hartree term 
((f = 0) docs not contribute, since the average of /(q) over 
a 4tt solid angle is zero. The Fock term {q=k — k') can 
be expanded into angular momentum channels once the 
magnitudes of k and k' are set to the Fermi wavevector 
kp {a good approximation in the low energy theory of a 
Landau FL [9 ): 



f{k-k') = - J2 /' 



Yg*{k)Yg,^^,{k'). (3) 



The minus sign originates in the Fock term. The dimen- 
sionless coupling constants of this expansion, Fg^m-.e'.m' = 
Fi'^m';£,m = N (Q) f e,rn;i' ,m' are the generalized Landau 
parameters, where iV(0) is the density of states at the FS. 
Because the rotational symmetry is broken by the polar- 
ization. Landau parameters Fi^m;t,m' that mix different 
angular momentum channels enter in the Hamiltonian. 
For the DDI, the non-vanishing Landau parameters are 
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More generally, the Fg^m-jj .m' will be renormalized by in- 
teractions beyond the DDI, such as the contact inter- 
action, but rotational symmetry and space-inversion (or 
time-reversal) symmetry of the system enforce the selec- 



tion rules 



and I — ^ = even integers. 



We now derive the free energy F in order to 

show that there is a phase transition to a biaxial state. 
In the sprit of the Landau theory of phase transitions, 
we expand in powers of the order parameters 

\ug^rn\- This procedure is accurate if the FS distortion 
is small. Since the uniaxial distortion — parametrized by 
the order parameters U2n,o — is present for all values of 
A, this expansion is quantitatively accurate only for A <C 
1, although it is qualitatively reasonable even for A ~ 
1. Additionally, the expansion in powers of the biaxial 
order parameters U2n,m {"ti 7^ 0) is accurate near the 
critical value A^ for the phase transition to the biaxial 
state. A perturbative (Hartree-Fock) calculation of the 
self energy of a DDFG at T = yields a fermion self- 
energy of T,Hp{k) = CT(|fc|)P2(cos 6*^), where a{\k\) > 
is a monotonic function of |fc|. This result is consistent 
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with Eq. ([ij and implies that the FS is stretched along 
the z-direction with the symmetry I = 2 for any finite A. 

To determine the structure of the free energy while 
keeping all order parameters obfuscates the essential 
physics. We work instead with a truncated, though suf- 
ficient, set of modes: the uniaxial order parameters U2,o 
and U4_o, and the biaxial order parameters W2,±2 = "^2^2; 
which can be represented in terms of an amplitude |m2,2| 
and an (arbitrary) phase ip2- 

The free-energy density of the uniaxial state, at fixed 
particle density and in mean-field theory up to quadratic 
order in M2 0, has the form: 



T <^Tf, the coefficients A and g are 
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where F is the free energy, V is the volume of the sys- 
tem and N{0) is the density of states at Fermi level. In 
Eq. (|6|, the term linear in U2fi arises from the mixing 
between uo,o (i-e., the particle density n) and 1*2, Oi and 
exists because i^2,0;0,o 7^ due to the anisotropic nature 
of the interaction. For T <C Tp and weak coupling 0{X^), 
the dimensionless constants /12 and m2 are 
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with similar (but smaller) coefficients for U4.0 and higher 
channels, including the mixing terms with U2,o- Within 
this approach, a "meta-nematic" first order transi- 
tion |13} 123] (i.e., a discontinuous jump in the uniaxial 
distortion U2fi driven by A) does not occur in a 3D DDFG 
since m2 > f33^. 

By minimizing the free energy of Eq. ^ , the expecta- 
tion value of U2,o (up to 0{X^) and T <C Tp) is 
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Hence, the FS has a uniaxial distortion in the d-wave 
channel (2,0) as well as in higher even angular momen- 
tum channels, e.g., (4, 0). For both Dy and KRb, the dis- 
tortions in higher angular momentum channels are small, 
e.g., |it4^o|/|^^2.o| ~ 10~^. We numerically determined the 
functional temperature-dependence of W2,o and u^^ to be 
monotonically decreasing. 

We now demonstrate the existence of a phase transi- 
tion to a biaxial nematic state above a critical value of the 
coupling constant Ac. The leading instability to the FS 
occurs in the U2,±2 channels, which possess free-energy 
densities of 
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and is independent of the phase (p2 as expected by the 
xy azimuthal symmetry. In a small A expansion and for 
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where m is the (effective) mass of the fermions. We 
only include the leading order contributions to A and ne- 
glect quantitatively important effects due to, e.g., mixing 
between different angular momentum channels such as 
(2, 0) and (2, 2). Extending the present order of approxi- 
mation requires either extensive numerical computations 
or the extrapolation of a longer series expansion [24j . 

The sign of A dictates two different phases. For A > 0, 
the FL phase with an uniaxial distortion (u2,o) is sta- 
ble, and U2,±2 = 0. For A < the system is in 
a biaxial nematic phase, with |u2,2| — ^| A|/2(7. For 
A < Ac(T) and A > 0, the uniaxial nematic phase dis- 
cussed above is stable. For A > Xc{T) (stronger DDI), 
the free energy of Eq. ( 10 1 predicts a biaxial phase. 



The uniaxial-biaxial phase boundary, shown in Fig. [T] 
is T/Tp = {2^/6/n) ^\ — \c/\ where we estimate the 
QCP at Ac — 24/77r. The actual value of Ac may be 
larger than the mean field value predicted above due to, 
e.g., quantum fluctuations. Nevertheless, signatures of 
this phenomenon could be within range of current exper- 
imental investigations. 

While the largest magnetic moment among the atoms 
(specifically, Dy and Tb with 10 hb) provides a A^y — 
0.006 at n = 10^^ cm~^, the fermionic polar molecule 
KRb, which has recently been laser cooled to near degen- 
eracy, possesses a ground state electric dipole moment 
of 0.57 D [1]. This yields realizable Ac's of magnitude 
[AKRb,ADy] = [0.17,0.006] at n = lO^^ ^^-z a more 
interesting case is that of fermionic LiCs, which would 
possess the large dipole moment of ~5.5 D |26] and a 
•^LiCs ~ 18 for similar densities. Other inhomogeneous 
phases may occur for such large values of A. Despite 
rapid progress in cooling bosonic LiCs to its rovibronic 
ground state |27j , fermionic LiCs is relatively unexplored 
to date. Confinement would not substantially modify 
the formation of a biaxial state unless the trap is highly 
anisotropic. Inhomogeneous effects in an harmonic trap 
induce growth of biaxial state formation from the center 
outward. 

It is well known [28J that 3D mean field theory fails 
to predict the correct critical behavior even though it 
yields a good qualitative picture. Further studies using 
large-scale numerical simulations will be needed to refine 
this picture. In particular, mean field theory does reveal 
that the biaxial state manifests itself as a FS distortion 
along two orthogonal axes, with the in-plane distortion 
given by |m2,2| and it spontaneously breaks the 5*0(2) 
rotational symmetry of the system (up to in-plane rota- 
tions by tt). In the case of a DDFG confined in a very 
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FIG. 1: The uniaxial-biaxial phase transition; Ac — 

large and isotropic trap, the correct universality class of 
the thermal uniaxial-biaxial transition we discuss here is 
the 3D classical XY model. Like all nematics, the bro- 
ken symmetry is not internal but the symmetry of spatial 
rotation [29] with a fixed axis. Unlike the classical XY 
model — which involves an internal symmetry — the topo- 
logical defects of this biaxial phase are disclination lines 
that cannot form closed loops because of the presence 
of the polarization field. Although this effect does not 
change the critical behavior in 3D, it does affect the be- 
havior of the biaxial state. 

The uniaxial phase is essentially a FL state with 
well defined quasiparticles and an explicit static spatial 
anisotropy. The Fermi velocity and the speed of (zero) 
sound vary along the "meridians" of the FS, as can be de- 
termined directly from the uniaxial distortion. We have 
studied the collective modes in both phases and at the 
phase transition. In addition to zero sound (an s-wave 
collective mode), the uniaxial phase has three quadrupo- 
lar collective excitations, one with = and two with 
Lz = ±2. The Lz = mode mixes with the zero sound 
as a result of the explicit anisotropy. 

As the phase transition to the biaxial state is ap- 
proached, much as in the 2D nematic state [S], the in- 
plane quadrupolar collective modes become gapless and 
overdamped, but only when propagating in the basal xy- 
plane. This becomes a Goldstone mode in the biaxial 
phase but remains gapped when propagating off-plane. 
This behavior is expected since this biaxial state devel- 
ops in the presence of an external symmetry breaking 
field: the only spontaneously broken symmetry occurs 
in the rotations about the direction of polarization. In 
contrast, a full biaxial nematic should have two gapless 
Goldstone modes as it has two spontaneously broken con- 
tinuous symmetries [2T]. 

Beyond employing destructive density measurements 
via light absorption to measure FS distortions, we sug- 
gest the use of polarized light scattering [21] to detect 
the collective behavior of DDFGs, since in the biaxial ne- 
matic state the DDFG behaves as a birefringent medium 
for light propagation. While this is a conceptually natu- 
ral and direct way to access the collective mode spectrum 
of the nematic phase, experimentally it may be challeng- 
ing to perform without destroying the nematic state. In 
particular, only small scattering signals may be present 
due to the need for limiting the atomic excited state 



population while interrogating the small trap popula- 
tion. An alternative method would involve measuring the 
anisotropy of scattering Bragg peaks; structure factors 
of non-dipolar Fermi gases were recently measured using 
Bragg spectroscopy (see Ref. [3D] and citations within). 

Nematic Fermi fluids — in the absence of a background 
lattice or at quantum criticality — are a striking exam- 
ple of a "non-Fermi liquid," in the sense that the quasi- 
particles are generally broad and poorly defined [9, 31]. 
However, in the case of this biaxial state, the quasipar- 
ticles become broad and non- Fermi liquid-like only when 
they propagate in-plane due to the polarizing field. An 
exciting issue is the possibility for realizing a true biax- 
ial nematic which would exhibit exotic defects known as 
non-Abclian disclination lines [32 . Such defects involve 
twisting polarization and have skyrmionic structure. 

In the context of a DDFG, a biaxial necessarily requires 
that the polarization be regarded as an order parameter, 
i.e., ferromagnetism. Thus, the biaxial phase of a DDFG 
is more complex than in conventional liquid crystals. In 
the absence of a ferromagnetic phase of 3D DDFGs, a 
true biaxial phase may appear as a metastable state ob- 
tained by turning off the external polarization field. Ho- 
mogeneously trapped DDFGs are thus a natural setting 
to investigate the possible existence of quantum liquid 
crystal phases in Fermi fiuids and will provide a fertile 
ground for explorations in non- Fermi liquid physics. 
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